Content Analysis to Enhance Voice Search

ABSTRACT

Methods and apparatus for improving speech recognition accuracy in media content searches are described. An advertisement for a media content item is analyzed to identify keywords that may describe the media content item. The identified keywords are associated with the media content item for use during a voice search to locate the media content item. A user may speak the one or more of the keywords as a search input and be provided with the media content item as a result of the search.

RELATED APPLICATIONS

The present application claims priority to, and is a continuation of,U.S. patent application Ser. No. 14/950,244, filed Nov. 24, 2015,entitled “Content Analysis to Enhance Voice Search”, which is herebyincorporated by reference in its entirety for all purposes.

BACKGROUND

Voice recognition systems can be useful tools for controlling acomputing system, but the usefulness of such a system is limited by thevocabulary that the voice recognition system can recognize. In somesituations, such as dealing with ever-changing media content (e.g.,television programs, movies, songs, etc.), the relevant vocabulary canbe difficult to establish, because of the wide variety of words andterms (and even unusual terms, like names) that may be used to refer tothat content.

In current systems, the difficulty with establishing a relevantvocabulary with which to describe media assets may result in a userbeing unable to find the media content that the user is searching for,because the user may not know the particular vocabulary used by a mediasearch system and/or media guide to refer to that media content. Thereremains an ever-present need for a media search system that allows theuser to search for content in a more natural manner.

SUMMARY

The following summary is for illustrative purposes only, and is notintended to limit or constrain the detailed description.

Aspects of the disclosure relate to apparatuses, computer-implementedmethods, and computer-readable media for determining keywords associatedwith a first media content, such as an audiovisual advertisement,determining that the first media content describes or relates to asecond media content, and associating the keywords with the second mediacontent. In aspects of the disclosure, the keywords may be determinedfrom audio, video, metadata and/or closed captioning portions of thefirst media content. Speech recognition may be used in determiningkeywords from the audio portion of the first media content. In addition,various online resources may be accessed for information to use indetermining the keywords. In some aspects, the keywords may be stored ina speech recognition database for use during a speech based search.

Other aspects of the disclosure describe a method for using speech asinput to a media item search. In some aspects, a speech utterance by auser may be recognized or otherwise converted to text or otherrepresentation. The converted utterance may be compared to keywordsassociated with media items, for example to keywords stored in a speechrecognition database and associated with media items, in order to locatea media item with one or more keywords corresponding to the utterance.In some aspects, a voice search may be used to locate a media item.

The preceding presents a simplified summary in order to provide a basicunderstanding of some aspects of the disclosure. The summary is not anextensive overview of the disclosure. It is neither intended to identifykey or critical elements of the disclosure nor to delineate the scope ofthe disclosure. The summary merely presents some concepts of thedisclosure in a simplified form as a prelude to the description below.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood with regard to the followingdescription, claims, and drawings. The present disclosure is illustratedby way of example, and not limited by, the accompanying figures in whichlike numerals indicate similar elements.

FIG. 1 shows an example communication network on which various featuresdescribed herein may be used.

FIG. 2 shows an example computing device that can be used to implementany of the methods, servers, entities, and computing devices describedherein.

FIG. 3 shows an example media search system on which features describedherein may be practiced.

FIG. 4 shows various data structures that may be generated according tothe features described herein.

FIG. 5 shows an example advertisement being analyzed by the media searchsystem to generate data structures to improve media content searchesaccording to features described herein.

FIGS. 6A and 6B show an example algorithm illustrating featuresdescribed herein.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which aspects of the disclosure may be practiced. It isto be understood that other embodiments may be utilized, and structuraland functional modifications may be made, without departing from thescope of the present disclosure.

The present disclosure describes several features of a voice recognitionsearch system, also referred to as a media search system. Advertisementsthat promote a product often use ideal keywords to describe theadvertised product, and it may be helpful to use those keywords to traina voice recognition search system to better identify if a user is askingabout the advertised product. For example, an advertisement campaign fora media content series such as “Ali G Rezurection” may use a particularphrase, e.g., “Da Ali G Show,” (a phrase used in the past to identifythe show) in describing the media content in its advertisements. Thepresent disclosure describes features of a voice recognition searchsystem that is able to associate that phrase with the media content. Inembodiments herein, the various advertisements for the “Ali GRezurection” television show may be preprocessed to identify thosephrases, and the phrases may be added to a database of keywords andphrases that are understood by a voice recognition search system torefer to the “Ali G Rezurection” media content. Media content or mediacontent items, as referred to herein, may include various types ofbroadcast television shows and movies, on-demand television shows andmovies, internet based videos, music videos, streaming videos, songs,podcasts, and any other media files.

The advertisements may vary in type, and each type may be processeddifferently to identify the keywords and phrases. For example, if anadvertisement is an audiovisual commercial, the system may extract thevideo content and the audio content and separately process the video andaudio content of the audiovisual stream to identify keywords andphrases. For example, keywords and phrases may be identified from theaudio content by performing a speech to text conversion of the audiocontent of the audiovisual commercial and identifying particularkeywords from the converted text of the audio stream. For example, anatural language processing (NLP) system may be able to identifyparticular words from the converted text of the audio stream to bekeywords for use in searching. The video portion of the audiovisualcommercial may be processed differently than the audio portion toidentify keywords. For example, the system may perform optical characterrecognition (OCR) processing of each frame of the video content of theaudiovisual stream to identify text in each frame. The resulting OCRtext from the video portion may be processed using an NLP system toidentify particular keywords. The system may remove duplicate keywordspresent in both the processed text of the audio and the video portionsof the audiovisual commercial.

As another example, if the advertisement is an Internet page with textand graphics, the system may extract all the keywords from the text. Thesystem may perform OCR on each graphic present in the Internet page andidentify keywords from any resulting text. The system may removeduplicate keywords present in the text and the graphics of the Internetpage.

As yet another example, if the advertisement is an audio advertisement(e.g., on a radio station or podcast), the system may perform a speechto text conversion of the audio content of the audio commercial andidentify particular keywords from the converted text of the audiostream.

In each of the examples above, the system may process the advertisementto identify keywords and phrases that refer to the “Ali G Rezurection”media content item. Those keywords and phrases may then be added to themetadata for the “Ali G Rezurection” media content item. By addingkeywords found in the advertisements and related media content promotingthe “Ali G Rezurection” media content item to that media content item'smetadata, the system may enrich the search database that is queriedduring a media content search. For example, by adding keywords to themetadata for media content items that is searched during a media contentsearch, the system may yield search results with higher accuracy if theuser searches for a particular media content item with keywordsdescribing the media content item that would otherwise not be present inthe title of the show or summary of the media content item.

In some embodiments, the media search system may use advertisements toidentify pertinent keywords that correspond to the content beingadvertised, and train a voice recognition system to associate thosekeywords with the content being advertised. The media search system mayanalyze each advertisement to determine whether the advertisement ispromoting a particular media content, or whether the advertisement isunrelated to and/or not promoting any media content that is accessibleto the media search system. The media search system may be able todistinguish advertisements from the rest of the media contentprogramming. Upon detecting advertisements, the media search system mayanalyze the media content of the advertisement to determine which mediacontent item (if any) the advertisement is promoting. If theadvertisement is determined to promote a media content item, the mediasearch system may further analyze the media content of the advertisementand extract keywords from the advertisement to add to the list ofvoice-recognizable keywords for the corresponding media content itembeing promoted. The addition of such keywords may help make it easierfor a user to use voice commands to ask for a particular media content,such as a television show, by training the system to recognize the wordsthat are used in advertisements for the media content.

In some embodiments, the media search system may also monitor user voiceinput to add keywords to media content metadata. By monitoring the uservoice input, the media search system may add keywords that users use todescribe media content items into the media content metadata for thecorresponding media content items. Voice input may be processed by aspeech recognition system to detect if the user is talking about aparticular media content. The voice input may be monitored to identifywords and phrases that the user uses to describe each particular mediacontent. For example, the user's voice input may be converted to a textstream using a speech to text conversion algorithm. The media searchsystem may process the text stream using NLP algorithms to identifykeywords in the user phrases that may be used by the user to describe amedia content item. Such keywords identified from the user's voice inputmay be stored in the metadata of corresponding media content items toimprove future voice searches.

FIG. 1 shows an example communication network 100 on which many of thevarious features described herein may be implemented. The network 100may be any type of information distribution network, such as satellite,telephone, cellular, wireless, etc. One example may be an optical fibernetwork, a coaxial cable network, or a hybrid fiber/coax distributionnetwork. Such networks 100 use a series of interconnected communicationlinks 101 (e.g., coaxial cables, optical fibers, wireless, etc.) toconnect multiple premises 102 (e.g., businesses, homes, consumerdwellings, etc.) to a local office or a headend. The local office 103may transmit downstream information signals onto the links 101, and eachpremises 102 may have a receiver used to receive and process thosesignals.

There may be one link 101 originating from the local office 103, and itmay be split a number of times to distribute the signal to variouspremises 102 in the vicinity (which may be many miles) of the localoffice 103. The links 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to help convey the signalclearly. Portions of the links 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths.

The local office 103 may include an interface 104, for example, atermination system (TS). More specifically, the interface 104 may be acable modem termination system (CMTS), which may be a computing deviceconfigured to manage communications between devices on the network oflinks 101 and backend devices such as the computing devices 105-106 andthe application server 107 (to be discussed further below). Theinterface 104 may be as specified in a standard, such as the Data OverCable Service Interface Specification (DOCSIS) standard, published byCable Television Laboratories, Inc. (a.k.a. CableLabs), or it may be asimilar or modified device instead. The interface 104 may be configuredto place data on one or more downstream frequencies to be received bymodems at the various premises 102, and to receive upstreamcommunications from those modems on one or more upstream frequencies.

The local office 103 may also include one or more network interfaces108, which can permit the local office 103 to communicate with variousother external networks 109. These networks 109 may include, forexample, networks of Internet devices, telephone networks, cellulartelephone networks, fiber optic networks, local wireless networks (e.g.,WiMAX), satellite networks, and any other desired network, and thenetwork interface 108 may include the corresponding circuitry needed tocommunicate on the external networks 109, and to other devices on thenetwork such as a cellular telephone network and its corresponding cellphones.

As noted above, the local office 103 may include a variety of computingdevices 105-106 and the application server 107 that may be configured toperform various functions. For example, the local office 103 may includea push server 105. The push server 105 may generate push notificationsto deliver data and/or commands to the various premises 102 in thenetwork (or more specifically, to the devices in the premises 102 thatmay be configured to detect such notifications). The local office 103may also include a computing device 106, which may be a content server.The computing device 106 may be one or more computing devices that areconfigured to provide content to users at their premises. This contentmay be, for example, video on demand movies, television programs, songs,text listings, etc. The computing device 106 may include software tovalidate user identities and entitlements, to locate and retrieverequested content, to encrypt the content, and/or to initiate delivery(e.g., streaming) of the content to the requesting user(s) and/ordevice(s). Indeed, any of the hardware elements described herein may beimplemented as software running on a computing device.

The local office 103 may also include one or more application serverssuch as application server 107. The application server 107 may be acomputing device configured to offer any desired service, and may runvarious languages and operating systems (e.g., servlets and JSP pagesrunning on Tomcat/MySQL, OSX, BSD, Ubuntu, Redhat, HTML5, JavaScript,AJAX and COMET). For example, an application server may be responsiblefor collecting television program listings information and generating adata download for electronic program guide listings. Another applicationserver may be responsible for monitoring user viewing habits andcollecting that information for use in selecting advertisements andproviding personalized media content recommendations to the user. Yetanother application server may be responsible for formatting andinserting advertisements in a video stream being transmitted to thepremises 102. Although shown separately, one of ordinary skill in theart will appreciate that the computing devices 105, 106, and/or theapplication server 107 may be combined. Further, here the computingdevices 105, 106, and the application server 107 are shown generally,and it will be understood that they may each contain memory storingcomputer executable instructions to cause a processor to perform stepsdescribed herein and/or memory for storing data.

An example premise 102 a, such as a home, may include an interface 120.The interface 120 can include any communication circuitry needed toallow a device to communicate on one or more links 101 with otherdevices in the network. For example, the interface 120 may include amodem 110, which may include transmitters and receivers used tocommunicate on the links 101 and with the local office 103. The modem110 may be, for example, a coaxial cable modem (for coaxial cable lines101), a fiber interface node (for fiber optic lines 101), twisted-pairtelephone modem, cellular telephone transceiver, satellite transceiver,local Wi-Fi router or access point, or any other desired modem device.Also, although only one modem is shown in FIG. 1, a plurality of modemsoperating in parallel may be implemented within the interface 120.Further, the interface 120 may include a gateway interface device 111.The modem 110 may be connected to, or be a part of, the gatewayinterface device 111. The gateway interface device 111 may be acomputing device that communicates with the modem(s) 110 to allow one ormore other devices in the premises 102 a, to communicate with the localoffice 103 and other devices beyond the local office 103. The gatewayinterface device 111 may be a set-top box (STB), digital video recorder(DVR), computer server, or any other desired computing device. Thegateway interface device 111 may also include (not shown) local networkinterfaces to provide communication signals to requestingentities/devices in the premises 102 a, such as display devices 112(e.g., televisions), additional STBs or DVRs 113, personal computers114, laptop computers 115, wireless devices 116 (e.g., wireless routers,wireless laptops, notebooks, tablets and netbooks, cordless phones(e.g., Digital Enhanced Cordless Telephone—DECT phones), mobile phones,mobile televisions, personal digital assistants (PDA), etc.), landlinephones 117 (e.g. Voice over Internet Protocol—VoIP phones), and anyother desired devices. Examples of the local network interfaces includeMultimedia Over Coax Alliance (MoCA) interfaces, Ethernet interfaces,universal serial bus (USB) interfaces, wireless interfaces (e.g., IEEE802.11, IEEE 802.15), analog twisted pair interfaces, Bluetoothinterfaces, and others.

FIG. 2 shows general elements that can be used to implement any of thevarious computing devices discussed herein. The computing device 200 mayinclude one or more processors 201, which may execute instructions of acomputer program to perform any of the features described herein. Theinstructions may be stored in any type of computer-readable medium ormemory, to configure the operation of the processor 201. For example,instructions may be stored in a read-only memory (ROM) 202, a randomaccess memory (RAM) 203, a removable media 204, such as a UniversalSerial Bus (USB) drive, compact disk (CD) or digital versatile disk(DVD), floppy disk drive, or any other desired storage medium.Instructions may also be stored in an attached (or internal) hard drive205. The computing device 200 may include one or more output devices,such as a display 206 (e.g., an external television), and may includeone or more output device controllers 207, such as a video processor.There may also be one or more user input devices 208, such as a remotecontrol, keyboard, mouse, touch screen, microphone, etc. The computingdevice 200 may also include one or more network interfaces, such as anetwork input/output (I/O) circuit 209 (e.g., a network card) tocommunicate with an external network 210. The network I/O circuit 209may be a wired interface, wireless interface, or a combination of thetwo. In some embodiments, the network I/O circuit 209 may include amodem (e.g., a cable modem), and the external network 210 may includethe communication links 101 discussed above, the external network 109,an in-home network, a provider's wireless, coaxial, fiber, or hybridfiber/coaxial distribution system (e.g., a DOCSIS network), or any otherdesired network.

In some embodiments, a media interface may be generated for display bythe processor 201 at the computing device 200, which may correspond to adevice local to a user, such as the set-box 113 as referenced in FIG. 1.In other embodiments, the media interface may be generated at anapplication server 107 at a local office 103 as referenced in FIG. 1. Inother embodiments, portions of the media interface may be generated atboth an application server 107 at the local office 103 and for displayby the processor 201 of the computing device 200.

The media interface may be displayed at the display 206. The processor201 may instruct device controller 207 to generate such a display at thedisplay 206. The processor 201 may receive user input to the mediainterface from input device 208. The processor 201 may process the userinput and implement subsequent features of the personalized media guideto such received user input. The processor 201 may store user mediaconsumption history, media preferences, and/or user profile informationin a memory unit such as ROM 202, RAM 203, or the hard drive 205. Theprocessor 201 may additionally identify any media content stored on thehard drive 205 or the removable media 204 and incorporate such locallystored media content into the personalized media guide. If such locallystored media content is requested for playback through the mediainterface, the processor 201 may retrieve such locally stored mediacontent from the removable media 204 or the hard drive 205 and displaythe locally stored media content on the display 206.

Additionally, the device may include a location-detecting device, suchas a global positioning system (GPS) microprocessor 211, which can beconfigured to receive and process global positioning signals anddetermine, with possible assistance from an external server and antenna,a geographic position of the device. The GPS microprocessor 211 maytransmit the determined location of the user of the computing device 200to processor 201. The processor 201 may then use the determined locationto further tailor the personalization of the media interface. Forexample, the processor 201 may identify users in the same location asthe user of the computing device 200 that have similar tastes as theuser of the computing device 200 based on a consumption history dataobtained from an application server 107. The processor 201 may generatecontent recommendations for the media interface displayed at the display206 based on the preferences of the identified similar users.

The FIG. 2 example is a hardware configuration, although the illustratedcomponents may be implemented as software as well. Modifications may bemade to add, remove, combine, divide, etc. components of the computingdevice 200 as desired. Additionally, the components illustrated may beimplemented using basic computing devices and components, and the samecomponents (e.g., the processor 201, the ROM storage 202, the display206, etc.) may be used to implement any of the other computing devicesand components described herein. For example, the various componentsherein may be implemented using computing devices having components suchas a processor executing computer-executable instructions stored on acomputer-readable medium, as illustrated in FIG. 2. Some or all of theentities described herein may be software based, and may co-exist in acommon physical platform (e.g., a requesting entity can be a separatesoftware process and program from a dependent entity, both of which maybe executed as software on a common computing device).

One or more aspects of the disclosure may be embodied in acomputer-usable data and/or computer-executable instructions, such as inone or more program modules, executed by one or more computers or otherdevices. Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types when executed by a processor ina computer or other data processing device. The computer executableinstructions may be stored on one or more computer readable media suchas a hard disk, optical disk, removable storage media, solid statememory, RAM, etc. As will be appreciated by one of skill in the art, thefunctionality of the program modules may be combined or distributed asdesired in various embodiments. In addition, the functionality may beembodied in whole or in part in firmware or hardware equivalents such asintegrated circuits, field programmable gate arrays (FPGA), and thelike. Particular data structures may be used to more effectivelyimplement one or more aspects of the disclosure, and such datastructures are contemplated within the scope of computer executableinstructions and computer-usable data described herein. The variouscomputing devices, servers and hardware described herein may beimplemented using software running on another computing device.

FIG. 3 shows an example media search system 300. The computing device302 may receive the media content 306 from the content server 304 overthe network 308. The computing device 302 may also communicate with oneor more information server(s) 332 to search Internet webpages or anyfiles stored on a remote database that are either related to and/orpromoting a media content item that is accessible to the media searchsystem 300. For example, the webpage and/or file promoting a mediacontent item may be a webpage devoted to particular media content itemsuch as a webpage for the “Ali G Rezurection” television show. Uponidentifying any content related to and/or promoting a media content itemon the information server(s) 332, the computing device 302 may gatherinformation related to the corresponding media content from theinformation server(s) 332. The computing device 302 may also receiveuser voice inputs from a microphone 334. The microphone 334 may beconnected directly to a microphone port of the computing device.Alternatively or additionally, the microphone 334 may a part of anothercomputing device that is in communication with the computing device 302over a wireless network such as Wi-Fi or Bluetooth. Upon analyzing mediacontent, the computing device may create data structures containingassociations between media content and advertisements, such as the datastructure 322 between the media content 324 and the advertisement 326.The data structure 322 may be a data structure that links the mediacontent 324 to an advertisement 326 that describes the media contentitem 324. The computing device 302 may further analyze theadvertisements to extract keywords describing the media content thatthey are promoting. The computing device 302 may generate an associationbetween each media content item and its corresponding keywords such asthe association 328 between the media content 324 and the keyword 330.

In some embodiments, the computing device 302 may perform multipledifferent features of advertisement recognition, parsing and analysis,speech recognition, and/or user utterance analysis. In otherembodiments, such functionalities may performed by more than onecomputing device of a distributed computing environment. In one exampleembodiment, the automatic speech recognition (ASR) engine 310 and theuser utterance detection engine 312 may be executed on one computingdevice in communication while the advertisement detection engine 314,the content analyzer 316, the keyword extraction engine 318, and thetrigram generator 320 may be executed on a second computing device incommunication with the first computing device. In another exampleembodiment, the ASR engine 310, the user utterance detection engine 312,the advertisement detection engine 314, the content analyzer 316, thekeyword extraction engine 318, and the trigram generator 320 may beexecuted on separate computing devices in communication with each other.In other examples, any combinations of these elements may be performedon any number of computing devices. For ease of reference, the featuresof the present disclosure will be discussed hereinafter as beingimplemented on a computing device 302.

In some embodiments, the computing device 302 may process voice commandsfrom the microphone 334 to translate spoken words to text input. The ASRengine 310 may receive audio input from the microphone 334. The ASRengine 310 may recognize voice inputs of users from other ambient noisesin the audio input and may convert the voice commands to text. Forexample, if the user says “Find me Da Ali G Show episode where Aliinterviews Buzz Aldrin,” the ASR engine 310 may recognize that the userhas input a voice command to search for a media content item and mayconvert the audio input of the voice command into text. The audio inputmay be converted to a text based input as soon as the ASR engine 310receives the audio input from the microphone 334.

In some embodiments, the computing device 302 may detect user utterancesfrom an audio input received from a user. A user utterance may be aportion of the voice command to search for a media content item. Forexample, “Da Ali G Show” and “Ali interviews Buzz” may be two userutterances present in the voice command “Find me Da Ali G Show episodewhere Ali interviews Buzz Aldrin.” Additionally or alternatively, theuser utterance may be detected separately from voice commands. The usermay be describing a particular media content item without issuing avoice command to search for a media content item. For example, the usermay be describing his favorite programs to customize the media searchsystem 300 to recommend relevant media content items. If the user states“I like the Ali G Show where Ali interviews Buzz Aldrin,” the computingdevice 302 may detect the user utterances of “Ali G Show” and “Aliinterviews Buzz Aldrin” in the user's voice input even though the user'svoice input is not a voice command since the voice input does not usewords such as “find me” or “show me” that are typical of voice commands.The computing device 302 may, however, identify the detected userutterances from such a user voice input to identify the media contentitem that the user is referring to in order to perform the intended taskfor which the voice input was received (e.g., improve the media searchsystem 300's media content recommendation algorithm). Such userutterances may be converted to text using a speech to text conversionalgorithm.

In some embodiments, the user utterance detection engine 312 may processthe text translated version of the audio input to identify the separateuser utterances in a continuous stream of audio input received from theuser. The user utterance detection engine 312 may identify if a user issearching for particular media content from other times the user istalking about something that is not related to searching for mediacontent. The user utterance detection engine 312 may identify if a userhas mentioned a particular media content item and may identify and storeall the other user utterances that the user mentions in relation to thatmedia content item. The user utterance detection engine 312 may beconfigured to monitor the translated speech-to-text input streamgenerated by the ASR engine 310 to identify if a user is talking about amedia content item. For example, the user utterance detection engine 312may continuously monitor a text stream outputted from the ASR engine 310for specific phrases that the user uses to refer to media content itemssuch as “show,” “television,” “series,” “episode,” “the one where,” etc.Detection of any of these phrases may trigger the user utterancedetection engine 312 to determine that the user is talking about aparticular media content item. For example, the user utterance detectionengine 312 may identify from the received voice command “Find me Da AliG Show episode where Ali interviews Buzz Aldrin,” that the terms “Show”and “episode” are included in the voice command. Accordingly, the userutterance detection engine 312 may identify that the user is talkingabout a media content item. Once the user utterance detection engine 312identifies that the user is talking about a particular media contentitem, the user utterance detection engine 312 may identify phrases thatthe user uses to refer to the media content the user is searching for.For example, the user utterance detection engine 312 may identify thatthe media content items. The user utterance detection engine 312 maymonitor the translated speech to text stream and may store words and/orphrases from the translated speech to text stream for each media contentitem that the user mentions. If the user says “the Rezurection episodewhere Ali G interviews an astronaut,” the user utterance detectionengine 312 may flag and/or store that portion of the voice input becauseit contains the phrase “episode where” which are typically used by theuser to refer to a media content item. The user utterance detectionengine 312 may store such phrases, hereinafter referred to as userutterances, in a memory of the computing device 302 to be relied uponfor searching through the metadata of different media content items insearch of a matching media content item.

In some embodiments, the computing device 302 may detect anadvertisement within the media content 306. For example, the mediacontent 306 may be a continuous media stream that includes advertisementsegments in between segments of the media content item. Theadvertisement detection engine 314 may be able to detect when anadvertisement begins and ends in such a continuous media stream. Theadvertisement detection engine 314 may receive a continuous stream ofprogramming from the content server 304 and accordingly, theadvertisement detection engine 314 may be able to detect which portionsof the continuous stream of programming are advertisements. For example,the advertisement detection engine 314 may analyze the metadata ofvarious different segments of the media content 306 received from thecontent server 304 to determine whether there are any identifiersindicating whether a given segment of the media content 306 is a mediacontent item or an advertisement. Additionally or alternatively, theadvertisement detection engine 314 may be able to detect advertisementsin a broadcast media stream from a content server 304 by detecting thelength of the programming, or identifying segments of programming basedon scene changes, the presence of blank frames that often begin and enda commercial, changes in audio level, or any other desired technique foridentifying different segments in a continuous media stream. Theadvertisement detection engine 304 may be able to determine that if themedia content lasts no longer than thirty seconds and may be positionedin the stream back to back with another such short duration mediacontent, then the media content is an advertisement. Additionally oralternatively, the advertisement detection engine 314 may detectadvertisements by detecting change in average volume of audio levels ofthe media content. If the audio volume is significantly higher for shortduration media content than its surrounding media content in acontinuous media stream, the advertisement detection engine 314 mayidentify the media content as an advertisement. The advertisementdetection engine 314 may also monitor the video and closed captioncontent of the media content to determine if there is continuous mentionto a particular brand or media content indicating that the advertisementis promoting such a brand of media content. Once the advertisementdetection engine 314 has identified an advertisement in the mediacontent 306, the advertisement detection engine 314 may mark the startand end times of each advertisement in media content 306. Theadvertisement detection engine 314 may store (e.g. in a memory of thecomputing device 302) a data structure including an association (e.g.,table or any other data structure) of all identified advertisementsrelated to a particular media content item and their associated startand end times for future reference. Additionally or alternatively, theadvertisement detection engine 314 may generate advertisements separatefrom media content 306 by extracting the advertisements in the mediacontent 306. Additionally or alternatively, advertisements may beidentified from an advertisement server. For example, the computingdevice 302 may communicate with an advertisement server to findadvertisements related to and/or promoting any media content itemsaccessible to the media search system 300. The computing device 302 mayanalyze any such identified advertisements in an advertisement server toextract keywords to aid in a voice media search for media contentaccording to the embodiments described herein.

In some embodiments, the computing device 302 may determine whetheradvertisements are promoting a media content item or whether they areunrelated to media content accessible to media search system 300. Thecontent analyzer 316 may analyze the audio and video data of the mediacontent that has been identified as an advertisement, by theadvertisement detection engine 314, to determine whether each identifiedadvertisement is describing and/or promoting a media content item. Thecomputing device 302, and in particular the content analyzer 316, mayextract text from advertisements and analyze the extracted text todetermine whether the extracted text includes terms describing any mediacontent items. If the advertisement is a video or an audiovisualcommercial, the computing device 302 may process the video component toextract any text present in the advertisement in order to perform such atext analysis. For example, the content analyzer 316 may perform opticalcharacter recognition (OCR) on each video frame of each identifiedadvertisement to identify any text displayed in the advertisement. Thecontent analyzer 316 may also retrieve closed caption informationassociated with each advertisement to identify terms used in theadvertisement. If the advertisement is an audio commercial (e.g., radio,or online radio commercial) or is an audiovisual commercial with anaudio component, the computing device 302 may translate the audiocomponent of the advertisement into a text stream to identify any termsindicating that the advertisement is related to and/or promoting a mediacontent item. For example, the content analyzer 316 may instruct the ASRengine 310 to process each identified advertisement's audio portion togenerate a text transcript of the advertisement using speech recognitiontechniques. Text extracted from performing OCR on the video frames of anaudiovisual and/or video advertisement, text obtained from the closedcaption information of advertisements, and text obtained from processingan audiovisual and/or audio advertisement's audio component using speechrecognition algorithms may be compiled into a text transcript of theadvertisement that may be analyzed to determine whether the identifiedadvertisement is promoting a media content and also to extract keywordsfrom the advertisement.

After text from the audio and video signals of an identifiedadvertisement has been generated, the content analyzer 316 may analyzesuch text to determine if the advertisement is related to any mediacontent item. For example, the content analyzer 316 may communicate withone or more information servers 332 to identify media contentinformation such as media guide data that include titles of media itemsand series, names for different media sources (e.g., broadcast channelsand on-demand providers), actor names, and other such media contentinformation that describes a media content item. The content analyzer316 may examine each advertisement (e.g., the generated text transcriptof an advertisement) for mentions of such media content information todetermine whether each advertisement is describing a media content itemor not. The content analyzer 316 may associate each advertisementpromoting a media content item with that media content item. The contentanalyzer 316 may generate one or more data structures including anassociation such as the data structure 322 that links a media contentitem 324 with all related advertisements that describe and/or promotethe media content item 324. For example, the content analyzer 316 maydetermine that the advertisement 326 contains an audio portion. When theaudio portion is translated into text by the ASR engine 310, the contentanalyzer 316 may detect that advertisement 326 includes mentions ofterms such as “Ali G Rezurection” and “FXX.” Upon communicating with oneor more information servers 332, the content analyzer 316 may determinethat the term “FXX” describes a content source and that the term “Ali GRezurection” describes a television show. Accordingly, the contentanalyzer may determine that the advertisement 326 is related to and/orpromotes the “Ali G Rezurection” media content item 324. Accordingly,the content analyzer 316 may generate the data structure 322 to includean association between the media content item 324 and the advertisement326. Such a data structure 322 that includes such an association may bestored in memory of the computing device 302 or in a separate computingdevice. The advertisements that the content analyzer 316 has determineddo not relate to any media content items may be excluded from a datastructure such as the data structure 322 associating media content itemswith advertisements.

As discussed above, the content analyzer 316 may process the texttranscript to determine, for example, that there are many references to“Ali G” in the advertisement, and that the advertisement is likely to bean advertisement for the “Ali G Rezurection” media content item. Anothercomponent, the keyword extraction engine 318, may process the texttranscript of advertisements to determine what kinds of keywords areused to describe the “Ali G Rezurection” media content item. Forexample, words such as “Sacha Baron Cohen,” “Da Ali G Show,” “Borat,”and “Buzz Aldrin” may be used in an advertisement 326 that promotes aparticular episode of the “Ali G Rezurection” television show series(e.g., media content item 324). The keyword extraction engine 318 mayextract words used in advertisement 326 such as such as “Sacha BaronCohen,” “Da Ali G Show,” “Borat,” and “Buzz Aldrin.” The keywordextraction engine 318 may receive the generated text transcript of eachadvertisement and analyze the text transcript to extract keywords. Thekeyword extraction engine 318 may ignore articles of speech, pronouns,conjunctions, and/or commonly used words in extracting keywords from thetext transcripts of each advertisement. The keyword extraction engine318 may be programmed with specific rules that govern how to extractkeywords (e.g., to identify and extract names of television channels andnames of movies, television shows, and music, actor names, etc.). Thekeyword extraction engine 318 may communicate with the informationservers 332 to identify such media content information (e.g., names oftelevision channels and names of movies, television shows, and music,actor names, character names, etc.) in order to know which words fromthe text transcript of advertisements to extract. For example, after theaudio portion is translated into text by the ASR engine 310, the keywordextraction engine 318 may detect that the audio portion of advertisement326 includes mentions of terms such as “Ali G.” Upon communicating withone or more information servers 332, the keyword extraction engine 318may determine that the term “Ali G” refers to a name of a character onthe “Ali G Rezurection” television show series and may extract this as akeyword. The keyword extraction engine 318 may be configured to place anemphasis on extracting proper nouns and to avoid extracting duplicatewords from the transcript. For example, the keyword extraction engine318 may extract the term “Buzz Aldrin” as the keyword 330 from the audioportion of the advertisement 326 upon determining that Buzz Aldrin is aproper noun. Keywords may be extracted from the advertisements 326 thathave been associated with each media content item 324. The computingdevice 302 may extract keywords from each of the multiple advertisements326 to generate the keywords 330 that the computing device 302 may storein a memory unit, either locally on the computing device 302 or remotelyin an information server 332. The computing device 302 may generate anassociation 328 between the media content item 328 and the keywords 330.

The computing device 302 may search for additional sources of keywordsand content describing media content items. By gathering keywords frommultiple different sources for each media content item, the media searchsystem 300 may capture different possible ways that people, especiallythe end users of the media content items, refer to the media contentitems. By searching through different online social networks formentions of various media content items, the media search system 300 maycapture keywords from posts in which one or more media content items arereferred to. The computing device 302 may analyze such posts to extractkeywords used by people to refer to such media content items that aredifferent from conventional keywords associated with the media contentitems from a media provider. By incorporating such keywords into themetadata searched for each media content item during a media search, themedia search system 300 may improve the media search process. Thecomputing device 302 may search for promotional content describing mediacontent items on the Internet or in a database stored on a local networkof the computing device 302 or on a network located remotely from thecomputing device 302. For example, the computing device 302 may searchfor feeds related to each particular media content item that the mediasearch system 300 has access to. Such feeds may be provided by mediacontent providers such as the content server 304 and/or be part of asocial networking website such as Twitter or Facebook. For example,keywords describing the media content may be extracted from onlinesocial networking services such as Twitter and/or Facebook. For example,messages posted on such online networking services may include ametadata tag such as a hashtag that may be used to identify whichmessages to parse to extract keywords for a media content item. Forexample, messages or posts on Facebook or Twitter with a metadata tag“#AliG” may be used to extract keywords about the media content itemtitled “Ali G Rezurection.” In some embodiments, the keyword extractionengine 318 may analyze any feeds received from the content server 304 toidentify if such feeds describe any media content items and if so,identify keywords from the description of each media content item. Forexample, the keyword extraction engine 318 may extract keywords from afeed provided by the content server 304 describing the media contentitem 324. The keyword extraction engine 318 may supplement the keywords330 with such keywords extracted from the feed. By doing so, suchkeywords extracted from the feed may be associated with each mediacontent 324 in a data structure 328.

Although advertisements are described throughout this disclosure assource of information from which to extract keywords describing mediacontent, any media content may be analyzed to obtain keywords describinganother media content. To comprehensively understand how people talkabout a particular media content, content from television shows, movies,user comments on online webpages related to the media content, userspeech detected from a microphone of a media consumption device, may beanalyzed to detect keywords that may be included as metadata describingmedia content items.

In some embodiments, keywords generated from user utterances (e.g.,words and/or phases detected from a user's voice input into thecomputing device 302) may be stored in a media content item's metadata.The content analyzer 316 may analyze user utterances identified by theuser utterance detection engine 312 to identify which user utterancesare related to media content items. By detecting words related to mediacontent items that the content analyzer 316 is configured to detect froma speech to text translation of the user's voice input, the contentanalyzer 316 may identify that certain user utterances may be describinga particular media content item. User utterances that the contentanalyzer 316 may have identified to describe a particular media contentmay be processed by the keyword extraction engine 318 to identifykeywords describing a media content item from such user utterances. Forexample, the system 300 may query a user to identify which shows theuser prefers and/or any particular shows that the user would like to seesimilar shows to. In response if the user responds with a user utterancesuch as “I liked the Ali G Rezurection episode where Ali G interviewsBuzz Aldrin,” the content analyzer 316 may identify that the user istalking about the “Ali G Rezurection” media content series byidentifying that the phrase “Ali G Rezurection” refers to media contentaccessible to the media search system 300. The content analyzer 316 mayfurther identify that user utterances “Ali G interviews Buzz Aldrin” mayrefer to an episode of the “Ali G Rezurection” television show and mayextract keywords such as “Ali G” and “interviews,” and “Buzz Aldrin”from the user utterance to tag the metadata of a “Ali G Rezurection”media content item.

In some embodiments, the computing device 302 may extract keywords byanalyzing the text translated voice command inputs for improved mediacontent search in the future. For example, if the user says “Find me theepisode where Ali G interviews Buzz Aldrin,” the content analyzer 316may analyze that voice command input and conduct a media search. Oncethe media search identifies that the media content item 324 is thesearch result corresponding to the user voice command, the computingdevice 302 may include the phrases “Ali G,” “interviews,” and “BuzzAldrin” extracted from the voice command input in the metadata for themedia content item 324 as keywords describing the media content item324. Adding such keywords to the metadata for media content items aftera media search has been conducted may enhance the degree of confidencein a media search if a future voice command input for the media contentitem 324 includes a different search phrase with one or more of thekeywords extracted in this manner.

The keyword extraction engine 318 may store keywords extracted from userutterances for a particular media content item in the metadata of thatparticular media content item. For example, the keyword extractionengine 318 may associate keywords extracted from user utterancesdescribing the media content 324, such as the keywords 330, with themedia content 324 and store such an association in the data structure328. The computing device 302 may store such a data structure 328 in amemory unit.

In some embodiments, associations such as those in the data structure328 between media content items and extracted keywords may be includedin each media content item's metadata. For example, the computing device302 may store associations such as the association between the mediacontent item 324 and the keyword 330 that is present in the datastructure 328 for in the metadata for the media content item 324. Themetadata may include data identifying the associations, keywords such asthe keyword 330 extracted from advertisements and/or media contentdescribing the media content item, and user utterances related to themedia content item.

The computing device 302 may store such data structures for each mediacontent item accessible to the media search system 300 in the respectivemedia content items' metadata. Keywords extracted from advertisementsmay be included in the metadata that already describes each mediacontent item. Keywords such as the keywords 330 may be included in themetadata that is used to search for media content items if a userinitiates a text or voice command search.

In some embodiments, trigrams may be generated from the keywords 330that are extracted from advertisements and media feeds. For example, atrigram generator 320 may generate various clusters of three keywords,hereinafter referred to as a keyword trigram. The three keywords thatare used in each keyword trigram may be selected from a list of allkeywords associated with a given a media content item 324. Variousdifferent combinations and/or permutations of three keywords associatedwith a particular media content item may be selected to generate suchkeyword trigrams. Keyword trigrams may be generated by examining keywordphrases. For example, for a keyword phrase “Da Ali G Show,” a word leveltrigram of “Da Ali G” or “Ali G Show” may be generated. Alternatively oradditionally, words from multiple keyword phrases may be used togenerate a word level keyword trigram. For example, a keyword phrases“Da Ali G Show” and “Rezurection” may be used to generate a keywordtrigram “Ali G Rezurection.” Such keyword trigrams may be stored alongwith media content metadata in order to effectively search for mediacontent items with voice commands. In some embodiments, the trigramgenerator 320 may generate keyword trigrams by selecting keywords thatare found nearby one another in the original source from which thekeywords have been extracted. For example, the trigram generator 320 maydetect that the keywords “Borat,” “interviews,” and “Buzz” occur neareach other in an original feed from which they have been extracted(e.g., Twitter feed for the Ali G Rezurection media content series). Bydetecting that such keywords appeared originally as a phrase “Tonight,watch Borat interview Buzz Aldrin on Ali G Rezurection,” the trigramgenerator 320 may determine that the keywords “Borat,” “interviews,” and“Buzz” are originally located near one another and may cluster themtogether to generate a word level trigram (e.g., “Borat interviewsBuzz”).

Trigrams may be used in the media search system 300 to improve theaccuracy of search results to a user voice command to search for mediaassets. By resolving the text translated voice command into trigrams andusing such voice command trigrams to search against a database ofkeyword trigrams that have been generated for keywords describing eachmedia asset, the accuracy of a voice media search may be improved. Insome embodiments, the keywords describing each media content item may bebroken down into clusters of three characters, hereinafter referred toas character level keyword trigrams. Any cluster of three characters maybe hereinafter referred to as a character level trigram, whereas anycluster of three words may be a word level trigram. The trigramgenerator 320 may generate character level trigrams from each keyword330.

Three consecutively placed characters from each keyword 330 or keywordphrase may be selected to generate a character level trigram comprisingthree characters that preserve the ordering in which such characters areplaced in the original keyword or keyword phrase. For example, from thekeyword phrase “Da Ali G Show,” the following character level trigramsmay be generated: Da_, _Al, Ali, i_G, Sho. By generating character leveltrigrams, the trigram generator 320 may determine which charactercombinations of keywords should be used to generate trigrams and whichcombinations should not be used. For example, while the trigramgenerator 320 may generate the character level trigram “Ali,” it may notuse the character combination of “_G_” to generate a trigram becausesuch a combination may not be determined to be of much value inidentifying or characterizing a media content item. In some embodiments,the trigram generator 320 may be configured to ignore spaces in keywordphrases and include three alphabetic characters when generatingcharacter level trigrams. For example, the trigram generator 320 maygenerate the character level trigram “li_G” from the keyword phrase “DaAli G Show” by ignoring the space character between “li” and “G” in thekeyword phrase “Da Ali G Show” and only selecting three consecutivelyplaced alphabetic characters in that phrase. However, the trigramgenerator 320 may maintain the space character in the character leveltrigram between the “li” and “G” even though it yields four totalcharacters in the trigram. In another implementation, the trigramgenerator 320 may remove the space character in the generated characterlevel trigram, resulting in the character level trigram “liG” havingonly three characters that are each alphabetic characters. In someembodiments, the word level trigrams and character level trigramsgenerated from advertisements, media feeds, and user utterancesdescribing a particular media content item may be included in the searchmetadata for the respective media content item. Such metadata may allowthe user to search for media content items by describing media contentitems in a natural manner instead of having to remember the names andtitles of episodes, media series, actors, or channel names to perform amedia search.

While trigrams may be generated for the various media content items'metadata, trigrams may also be generated for user utterances and/orother user voice inputs. By generating trigrams for both user voiceinputs and for keywords stored in a media content items' metadata, thecomputing device 302 may be able to perform a media content search bysearching through keywords in the metadata of various media contentitems using the trigrams generated from the text translated user voiceinput. The trigram generator 320 may identify user utterances generatedfrom a voice input received from the user. The trigram generator 320 mayreceive as inputs, user utterances generated from the user utterancedetection engine 312, and may generate word level trigrams and characterlevel trigrams of these received user utterances. Such user utterancetrigrams may be used to search for media content items as describedfurther below with relation to FIG. 4. The user utterance trigrams foruser utterances that describe a media content item may be grouped alongwith other keyword trigrams describing that media content item.

Although the embodiments described in this disclosure have beendescribed in the context of trigrams, any sized n-grams may be used inthe place of trigrams for word level and character level n-grams.Trigrams may be preferred over other n-grams for certain applicationssuch as media content item names and descriptions. The computing device302 may be configured to use a different size n value. For example, thecomputing device 302 may use bigram (cluster of two) or quadgram(cluster of four) words or characters in the media search process. Thecomputing device 302 may determine based on the average character and/orword count of each of the keywords stored for the media content items inthe media search system 300 that trigrams may be the most efficient sizen-grams to use for searching through such keywords to find a matchingmedia content item in a voice search. In some embodiments, if theaverage character count and/or word count of each of the keywords inmedia search system 300 is smaller than the average character countand/or word count of each of the keywords for which a trigram conventionis used, the computing device may be configured to use bigrams instead.

FIG. 4 shows various data structures that may be generated to facilitatethe natural language search for media content items in a media contentsearch system 400. As described in FIG. 3, an initial set of keywordsdescribing a media content item may be generated before a voice searchfor media content is conducted. For example, the trigram generator 406may generate content keyword trigrams such as the content keywordtrigrams 426 and 436 for the different media content items 422 and 432,respectively before search input is received from a user. Media contentitems such as the media content items 422 and 432 may have media contentmetadata associated with them, such as the media content metadata 424and 434. By finding advertisements, feeds, and parsing additionalcontent describing media content items, keywords and content keywordtrigrams may be generated for various different media content items andincluded in the metadata of such media content items. Such initialpreprocessing of media content keywords and content keyword trigrams mayoccur before a search input is received from a user. When a user inputsa voice command, such as search input 402, to perform a media search,the voice command may be translated into text and user utterances suchas the user utterances 412 a, 412 b, 412 c, 412 d, 412 e, and 412 f maybe generated from the text translation of such a search input receivedfrom a user. User utterance trigrams such as the user utterance trigrams414 a, 414 b, and 414 c may be generated from the user utterances 412a-f by a trigram generator 406. In order to implement a voice commandsearch for media content items, a search engine 404 may search contentkeyword trigrams using user utterance trigrams to find a match between amedia content item and the search input 402 received from a user. Avoice command input including the search input 402 may be received fromthe user in step 614 of method 600, discussed further below with regardto FIG. 6A.

The user utterances 412 a-f may be generated from a search input 402. Asdescribed in connection with step 616 of method 600 described in FIG. 6Abelow, a voice command search input 402 may be processed into a textstream by an ASR engine such as the ASR engine 310 as described abovewith relation to FIG. 3. The text stream may be analyzed to determineseparate user utterances that each describe a media content item. In theexample shown in FIG. 4, the user utterances 412 a-f are generated bymonitoring the text stream generated from voice commands comprised bythe search input 402. The user utterances 412 a-f may be saved in amemory unit of the media content search system 400 in response todetermining that such user utterances describe a media content item. Forexample, the user utterances 412 a-f may be extracted from the textstream generated from the user input 402 upon determining that suchwords are uttered in the same context as a media content item that themedia content search system 400 supports. By comparing certain words inthe text stream against a library of search terms known to be mediacontent search keywords, text from the search input 402 may beidentified to be related to media content searches. By determining allof the words related to each such identified text in the search input402, each of the user utterances 412 a-f may be identified as being userutterances related to a media content search. In some embodiments, eachuser utterance may include a single phrase that represents a user searchfor a particular media content item. In the example shown in FIG. 4,each user utterance in the user utterances 412 a-f may be related to asingle media content item. For example, each user utterance may be aphrase that the user utters to describe a media content item that theuser is searching for. In some other embodiments, each user utterancemay be related to a different media content item than another userutterance.

In some embodiments, user utterance trigrams such as the user utterancetrigrams 414 a-c may be generated from the user utterances 412 a-f. Suchuser utterance trigrams may be generated in step 618 of method 600,discussed further below with regard to FIG. 6A. In the exampleembodiment shown in FIG. 4, multiple user utterances may be used togenerate each user utterance trigram. In the example shown in FIG. 4,each user utterance 412 a-f may comprise one or more words. The userutterance trigram 414 a may be a word level trigram generated bycombining terms from the user utterances 412 a-c. Similarly, the userutterance trigrams 414 b and 414 c may combine terms from the userutterances 412 c-e and the user utterances 412 d-f, respectively. Suchuser utterance trigrams may be used by the search engine 404 to searchfor media content items. The search engine 404 may compare userutterance trigrams with content keyword trigrams such as the contentkeyword trigrams 426 and 436 to determine that there is a match betweenwhat the user is searching for and a media content item.

Each media content item may have associated metadata stored in thememory unit of the media content search system 400. For example, in theexample embodiment shown in FIG. 4, the media content items 422 and 432may have the media content metadata 424 and 434, respectively stored insuch a memory unit. The media content metadata 424 may comprise multiplekeywords such as the keywords 424 a, 424 b, and 424 c, which may havebeen stored after being extracted from advertisements, media feeds, orthe user utterances describing media content 422. Similarly, the mediacontent metadata 434 may comprise multiple keywords such as the keywords434 a, 434 b, and 434 c, which may have been stored after beingextracted from advertisements, media feeds, or user utterancesdescribing the media content 432. The trigram generator 426 may generatethe content keyword trigrams 426 and 436 from the media content metadata424 and 434, respectively. The content keyword trigrams 426 a and 426 bmay be word level trigrams and/or character level trigrams of thekeywords 424 a, 424 b, and/or 424 c. Similarly, the content keywordtrigrams 436 a and 436 b may be word level trigrams and/or characterlevel trigrams of the keywords 434 a, 434 b, and/or 434 c.

In order to search for media content items that the user may besearching for, the metadata for each media content may be searched usinguser utterance trigrams. For example, the search engine 404 may searchtrigrams of content keywords such as the content keyword trigrams 426 a,426 b, 436 a, and 436 b using the user utterance trigrams 414 a-c. Sucha search may be performed in step 620 of method 600, discussed furtherbelow with regard to FIG. 6A. The search engine 404 may determine thatthere is a match after a threshold number of matches between userutterance trigrams and content keywords trigrams for a given mediacontent is reached. Once such a threshold number of trigram matches isreached, the search engine 404 may determine that the manner in whichthe matching media content item is described in advertisements, mediafeeds, and/or user utterances matches the manner in which the user hasdescribed the desired media content item. Accordingly, a trigram matchbetween user utterance trigrams and content keyword trigrams generatedfrom metadata of each media content item may allow a user to search formedia content items in a more natural way instead of having to rely on akeyword match for the title of the media content item or media series orchannel number, each of which the user may not know.

FIG. 5 shows an example media search system 500 with an exampleadvertisement 502. The advertisement 502 may be a commercial displayedduring a television program or other video program, an advertisement onan Internet page, an advertisement sent or received in an email, aprinted paper or poster advertisement being scanned, etc., beinganalyzed by a media search system 500 to identify additional keywords orphrases that should be recognized as referring to the “Ali GRezurection” show. The media search system 500 may identify theadvertisement 502 from either an advertisement database or from a linearmedia content stream. Such an advertisement may be identified in step602 of method 600, discussed further below with regard to FIG. 6A. Themedia search system 500 may identify media content advertisements, suchas a television program advertisement, a movie advertisement, etc., asopposed to other types of advertisements that do not promote a mediacontent item accessible to the media search system 500.

In some embodiments, the media search system 500 may search foradvertisements promoting media content items. The media search system500 may identify an advertisement 502 from an advertisement database orit may identify the advertisement 502 from a linear stream of mediacontent broadcast from a media content server to a user device. Forexample, a computing device such as the computing device 302 of FIG. 3may detect that a portion of a linear media stream received from a mediacontent server may include one or more advertisements. Advertisementsmay be detected by analyzing the metadata and/or closed captioninformation associated with the media content for identifiers indicatingthe presence of an advertisement. Advertisements may be detected byanalyzing volume levels and durations of media clips in the linear mediastream. Once the advertisement has been identified, start and end timesof each advertisement in the media content linear stream may beidentified and noted in a memory of the computing device.

The advertisement 502 may be selected upon determining that theadvertisement 502 is an advertisement promoting a media content item,such as a television program advertisement, a movie advertisement, etc.as opposed to advertisements for products that are not media contentitems (e.g., advertisements for medicines, consumer products, etc.). Theaudio signal 504 of the advertisement 502 may be converted to a texttranscript using an ASR engine. A content analyzer of the media searchsystem 500 may analyze the audio signal 504 of the advertisement 502 andidentify the script of the advertisement 502. For example, the contentanalyzer of the media search system 500 may identify portions of theaudio transcript of the advertisement 502 that describe and/or promote amedia content item (e.g., a television program, movie, etc.).

In some embodiments, the advertisement 502 may include text describingthe name of the media content item that the advertisement 502 ispromoting. If the name of the media content item that the advertisement502 is promoting is included in the advertisement 502, the media searchsystem 500 may identify the text describing the name of the mediacontent item and may associate the advertisement 502 with that mediacontent item. The advertisement 502 may include the on-screen graphic506 such as channel identifiers that may comprise informationidentifying one or more characteristics of the media content that theadvertisement 502 is promoting and/or the advertisement itself. Theon-screen graphics 506 may describe which channel the advertisement 502may be received on by the user. Such on-screen media identifiers, alongwith text in the advertisement 502 that may identify the name of themedia content item it is promoting may be used to identify which mediacontent item the advertisement 502 may correlate to. Once theadvertisement 502 has been identified to promote a particular mediacontent item, the advertisement 502 may be associated with theparticular media content item. Such determination of whether anadvertisement describes and/or is related to a media content item may beperformed in step 604 of method 600, discussed further below with regardto FIG. 6A.

In some embodiments, the advertisement 502 may not include the name ofthe media content item that it is promoting. In order to identifywhether the advertisement 502 is related to any media content items, theadvertisement 502's on-screen media identifiers, text transcript of theadvertisement audio, and closed captions may be used to identify whichmedia content item the advertisement 502 may correlate to. For example,by analyzing the on-screen graphic 506, and comparing it to a pre-storeddatabase of graphics and logos commonly used with media items and/orproviders, it may be identified that the particular media content thatthe advertisement 502 is promoting may be broadcast on FXX. Theadvertisement's audio component may also be analyzed by the contentanalyzer for words and/or sounds that are commonly used by media itemsand/or providers by searching a pre-stored database of audio samplesused with media items and/or providers. By further analyzing the audiosignal 504 of the advertisement 502, a content analyzer may determinethat the advertisement 502 includes an audio signal with the name of themedia content item

Once the advertisement 502 has been determined to be related to aparticular media content item that is accessible to the media searchsystem, the advertisement 502 may be analyzed for keyword extraction.For example, the text transcript of the advertisement 502's audio 504may be analyzed to identify keywords that describe the media contentitem that the advertisement 502 promotes. Such text extraction fromadvertisements and identification of keywords from the extracted textmay be performed in steps 606 and 608, respectively, of method 600,discussed further below with regard to FIG. 6A. From such a texttranscript of the advertisement, content keywords may be generated whichmay be used to search for the media content item. Once the media contentitem that the advertisement 502 may be promoting has been identified,keywords extracted from the advertisement 502 may be used to supplementthe metadata of the identified media content item. In the example shownin FIG. 5, the keywords 514 c, 514 d, and 514 e may be extracted fromthe text transcript of the advertisement 502's audio. For example, aportion of the advertisement 502's audio may include the phrase, “Myman, Buzz Lightyear! Do you think man will ever walk on the sun? Whathappens if they went in winter when the sun is cold?' By analyzing sucha phrase from the audio 504's text transcript, the keyword 514 d (‘BuzzLightyear’) and the keyword 514 e (‘Man Will Ever Walk on Sun’) may beextracted. Similarly, the audio signal 504 of the advertisement 502 mayalso indicate that the advertisement 502 is for the show “Ali GRezurection.” Accordingly, the keyword 514 c (‘Rezurection’) may beextracted from the audio signal 504. Such keywords may be extracted fromthe audio signal 504 by determining that such words are unique to themedia content item and therefore may likely describe the media contentitem. By detecting such keywords, the media search system 500 mayidentify keywords for inclusion in the media content metadata 514 forthe “Ali G Rezurection” episode that the advertisement 502 isdescribing. By analyzing the transcript of the advertisement 502, themedia search system 500 may identify that such keywords 514 c-e may beused by users in referring to the “Ali G Rezurection” episode.Accordingly, such keywords 514 c-e may be added to the media contentmetadata 514 that is stored in a memory of the media search system 500.Associating such identified keywords with their corresponding mediacontent items may be performed in step 610 of method 600, discussedfurther below with regard to FIG. 6A.

In some embodiments, after the media content item that the advertisement502 is promoting has been identified, additional sources of mediainformation (e.g. websites, media databases, etc.) may be searched togather additional description of the media content item. Such additionaldescription may be used to extract additional keywords to supplement themedia content metadata 514 for the media content item. These webpagesmay be searched using the identified name of the media content item. Forexample, by searching an online database or multiple webpages using anonline search engine with the title of the media content item that theadvertisement 502 may be describing, additional webpages describing thecontent may be found from which additional keywords may be extracted tosupplement the media content metadata 514. For example, upon identifyingfrom the audio signal 504 that the media content item being promoted is“Ali G Rezurection,” the media search system 500 may search foradditional information on the Internet and/or in media guide databasesfor media content items titled “Ali G Rezurection,” or “Ali G.” Themedia search system 500 may also refine the online search by includingkeywords identified from the advertisement 502 in the search string. Forexample, the media search system 500 may add keyword “Buzz Lightyear” tothe online search.

Such a database search may result in webpages such as the webpages 508and 520. The webpage 508 may be an IMDB webpage for the “Ali GRezurection” media series that results from conducting a search with thetitle of the media content (“Ali G” or “Ali G Rezurection”). A morerefined search using the keywords 514 c-e may result in the webpage 520,a webpage for the episode described in the advertisement 520. Thewebpage 520 may be found after searching an online database with thekeywords 514 c (‘Rezurection’) and 514 d (‘Buzz Lightyear’). Byexamining the text 510 and 522 found in the webpages 508 and 520,respectively, the media search system 500 may identify additionalkeywords such as the keywords 514 a (‘Sacha Baron Cohen’), 514 b (‘DaAli G Show’), and 514 f (‘Technology’). Such additional keywords may bestored in the media content metadata 514 for the particular mediacontent item that the advertisement 502 is promoting.

In some embodiments, trigrams may be generated from the keywords storedas the media content metadata 514. The media search system 500 maygenerate the content keyword trigrams 516 from the keywords 514 a-fstored in media content metadata 514. Such content keyword trigrams maybe generated in step 612 of method 600, discussed further below withregard to FIG. 6A. In the example shown in FIG. 5, the content keywordtrigrams 516 a-d are word level trigrams of the media content keywords514 a-f and the content keyword trigrams 516 e-1 are character leveltrigrams of the media content keywords 514 b-c. Such content keywordtrigrams 516 may be used in the media search to identify the particularepisode of the “Ali G Rezurection” series if a user searches for such amedia content item. Certain trigrams may include terms from only onekeyword (e.g., the word level trigram 516 c uses only terms from thekeyword 514 e). However, some trigrams may use terms found in multipledifferent keywords (e.g., the word level trigram 516 d uses terms fromthe keywords 514 b, 514 c, and 514 d). Similarly, character leveltrigrams may also use characters from different keywords. The characterlevel trigrams 516 i-1 are trigrams of the phrase “Ali G Rezurection,” aphrase that is a combination of terms from keyword 514 b (‘Da Ali GShow’) and keyword 514 c (‘Rezurection’). The trigram 516 k combinescharacters from the keywords 514 b and 514 c.

The media search system 500 may identify keywords for each media contentitem that it has access to by parsing through advertisements promotingthe media content items and online databases describing the mediacontent items. The media search system may accordingly generate clustersof keywords and trigrams for each media content item in this manner.Such clusters may be stored in a memory unit of the media search system500 for use in performing media searches. Responsive to a user issuing avoice command to search for a media content item, the media searchsystem 500 may search through these different clusters of trigrams andkeywords that are associated with each media content item to determinewhich media content item best matches the user utterances and/or theuser utterance trigrams generated from the voice command.

In some embodiments, the media search system 500 may be able to minimizethe effects of incorrectly spelled homophones, both in user utterancesor in speech recognition processed text transcript of the advertisement502's audio signal 504. For example, it may be possible that one or morewords in the audio signal 504 of the advertisement 502 are incorrectlyconfused by the media search system 500 with a homophone whilegenerating the text transcript of the audio signal 504. For example, thekeyword 514 c may be incorrectly spelled as “Resurrection” if the audiosignal 504 is speech processed through an ASR engine. Such an incorrectspelling may occur more frequently if the words used in the mediacontent item are less commonly used in everyday language than theirhomophones. For example, the speech recognition algorithm may mistakenlyprocess “Rezurection” as “Resurrection” because the word “Resurrection”is more commonly used than “Rezurection.” Although the word“Rezurection” is the correctly spelled title of the media content itembeing indexed, speech recognition algorithms may commonly confuse thisword with the word “Resurrection.” While conventional system maypropagate such a confusion in the search process by returning the wrongmedia content item due to such a mistake in transcription of the mediacontent item title, the media search system 500 may overcome suchconfusion. For example, most conventional systems may incorrectly tagthe episode of “Ali G Rezurection” with the word “Resurrection” and mayreturn “Ali G Rezurection” if the user is searching for a differentprogram named “Resurrection” (a different fantasy drama televisionshow). However, media search system 500 may overcome such confusion bysearching the media content trigrams 516 of keywords identified from theaudio 504 of the advertisement 502 promoting the media content item. Bysearching through the different trigrams 516 a-1, especially thecharacter level trigrams 516 e-1, the media search system 500 may beable to match the correct media content item even if it includesincorrectly spelled homophones because the overall match between thetrigrams 516 with the user utterance trigrams may be high even with theincorrect spelling of certain keywords.

The media search system may also be able to minimize the effects ofincorrectly spelled homophones in speech recognized text translations ofthe user voice commands. By generating user utterance trigrams from thevoice command inputs of a user as described in FIG. 4, the media searchsystem may be able to overcome making any incorrect matches between theuser search input describing the media content item and the mediacontent item itself. By conducting such a media content search usinguser utterance trigrams and media content keyword trigrams, the mediasearch system may be able to reduce the weight of any one utteranceand/or keyword, thereby minimizing the chance that any one misspelledutterance or keyword may result in an incorrect match.

According to the example shown in FIG. 5, if the user searches for amedia content by issuing the voice command “Ali G show where Buzz isinterviewed” or “Rezurection episode in which Ali asks an astronaut ifman can walk on the sun,” the media search system 500 may generate wordlevel and/or character level user utterance trigrams from such searchinputs. The media search system 500 may also have identified keywordsfrom advertisements promoting the particular episode of the “Ali GRezurection” series that the user is searching for. Media contentkeyword trigrams such as the trigrams 516 may have been associated withthe metadata for such a media content item. Accordingly, the metadatafor each of the media content items that the media search system 500 mayhave access to may have been clustered with such media content keywordsand trigrams. When the user issues voice commands, the media searchsystem 500 may compare the resulting user utterances and/or userutterance trigrams against the media content keywords 514 and/or themedia content trigrams 516 to determine a match between the user-issuedvoice commands and media content items. Since the metadata for the “AliG Rezurection” episode may include keywords “Rezurection” (keyword 514c), “Buzz Lightyear” (keyword 514 d), “Man Will Ever Walk on the Sun”(keyword 514 e), the media search system 500 may determine that such amedia content item 512 is a match. Additionally or alternatively, sincethe media content item 512 may be associated with content keywordtrigrams such as “Ali Rezurection Buzz” (the word level trigram 516 d),“Man Walk Sun” (the word level trigram 516 c), “Ali” (the characterlevel trigram 516 i), and “i_G” (the character level trigram 516 j), themedia search system 500 may determine that the media content item 512 isthe best match for the user issued voice commands. Accordingly, themedia search system 500 may provide the user with the media content 512,or an indication of the media content 512, as the search result if theuser issues such voice commands.

Although the description of FIG. 5 and much of the disclosure describesthe media search system 500 being used to improve the search for mediacontent items, such as television shows and movies, the media searchsystem 500 may also be used for searching other items, such as consumerproducts, electronics, automobiles, etc. The media search system 500 maysearch through advertisements, social media networks (e.g., Facebook,Instagram, Twitter) to find content promoting a particular product(e.g., electronics, automobiles, etc.) that is included in a searchabledatabase. The media search system 500 may extract keywords describingthe product from the advertisements, social media posts, and otherpromotional content and include the extracted keywords describing theproduct as metadata describing the product in a searchable database. Themedia search system 500 may also generate trigrams from the extractedkeywords as described above and index them in a searchable database.When a user conducts a product search, the media search system 500 maysearch through the database of extracted keywords and trigrams to findan accurate match for the product that the user is searching for, asdescribed throughout this disclosure. In this manner, the media searchsystem 500 may extract keywords describing the product that aredifferent from the formal product description but may instead reflecthow the general public colloquially refers to the product. By addingkeywords of this nature to a searchable database's metadata, the mediasearch system 500 may improve searching for various types of consumerproducts.

FIGS. 6A and 6B show an example method 600 according to the features asdescribed herein. The method may be performed by, for example, thevarious components in the systems shown in FIGS. 1, 2, and 3 or anyother computing device. The method may be implemented in a networkenvironment having devices such as an application server 107, a user'sconsumption or access device (e.g., the display device 112, the gatewayinterface device 111, the personal computer 114, the wireless device116, etc.), or any other desired computing device.

In step 602, an advertisement may be identified. An advertisement mayeither be identified from an advertisement database or it may beidentified from a linear stream of media content broadcast from a mediacontent server to a user device. For example, a computing device such asthe computing device 302 of FIG. 3 may detect that a portion of a linearmedia stream received from a media content server may include one ormore advertisements. Advertisements may be detected by analyzing themetadata and closed caption information associated with the mediacontent for identifiers indicating the presence of an advertisement.Advertisements may be detected by analyzing volume levels and durationof media clips in the linear media stream. Once the advertisement hasbeen identified, start and end times of each advertisement in the mediacontent linear stream may be identified and noted in a memory of thecomputing device. Additionally or alternatively, advertisements may beidentified by communicating with an advertisement database.

In step 604, the advertisement may be analyzed to determine whether itdescribes a media content item. Each advertisement from an advertisementdatabase may be analyzed to determine whether it is related to a mediacontent item. For example, the metadata and content of the advertisementmay be analyzed to determine if the advertisement is promoting and/ordescribing a media content item. The advertisement may be a promotionaltrailer for a media content item scheduled to be broadcast at a futuretime. Additionally or alternatively, the advertisement may promote oneor more media series. The advertisement may be analyzed to determine ifit is has content that is related to such media content items such thatthe advertisement can be analyzed to identify keywords describing mediacontent items. Each advertisement may be examined for mention of mediacontent information that has been retrieved from a media guide database,to determine whether each advertisement is describing a media contentitem or not. The content analyzer 316 may associate each advertisementpromoting a media content item with that media content. Advertisementsor media content that do not promote and/or describe any media contentitems may be marked as such in a memory unit in order to not referencesuch an advertisement for media content extraction at any point in thefuture. If at step 604, the advertisement is determined to not describeany media content items, then the method 600 may proceed to identify adifferent advertisement for analysis at step 602. Each advertisementthat has been identified from the linear stream of media contentreceived from the media content server may be analyzed to determinewhether it describes any media content items. The identifiedadvertisements may be further processed according to steps 606-612 ofthe method 600.

In step 606, in response to determining that the advertisement describesa media content item, text may be extracted from the advertisement.Speech recognition may be performed on an audio signal of theadvertisement to generate a text transcript of the advertisement'saudio. Optical character recognition may be performed on a video signalof the advertisement to extract any text from the video of theadvertisement. Closed caption information and metadata from theadvertisement may be extracted to supplement the text transcript. Textextracted from each of such components of an advertisement may be usedto generate a complete text transcript of the advertisement. Such acomprehensive text transcript may be used to ensure that every aspect ofthe advertisement may be analyzed during keyword analysis.

In step 608, keywords describing the media content item may beidentified from the advertisement. Keywords describing the media contentitem may be identified from the text transcript generated from theadvertisement in step 606. Articles of speech, pronouns, conjunctions,and/or commonly used words in the text transcript may be ignored inextracting keywords from the text transcripts of each advertisement. Akeyword extraction engine may be programmed with specific rules thatgovern how to extract keywords. The keyword extraction engine maycommunicate with information servers to identify media contentinformation (e.g., names of television channels and names of movies,television shows, and music, actor names, etc.) in order to determinewhich words from the text transcript of advertisements to extract.Keywords extracted from each advertisement may be stored in a memoryunit, either locally on the computing device or remotely in aninformation server. Additionally or alternatively, media feeds frommedia content providers may be analyzed to extract additional keywordsabout the media content item described by the advertisement beinganalyzed. Once the media content item that is being promoted has beenidentified, additional sources of information such as online webpagesand media feeds for that article may be searched and keywords describingthe identified media content item may be identified.

In step 610, the keywords identified from the advertisement may beassociated with the media content item. Once the keywords describing amedia content item have been identified from the advertisement and/orfrom other sources such as media feeds and online databases, thekeywords may be associated and/or clustered with the media content itemthat they are describing. For example, the keywords that have beenidentified to describe each media content item may be stored as metadataassociated with that media content item. Such metadata may be stored ina database for later use in a media content search. Such a database mayinclude keywords describing several different media content itemsgrouped according to the media content item that they each describe.

In step 612, trigrams may be generated from the keywords associated withthe media content item. For example, each of the keywords that have beenassociated with a particular media content item may be used to generatekeyword trigrams. Word level and character level trigrams may begenerated from the collection of keywords associated with each mediacontent item. For example, a cluster of keywords may be identified byretrieving all associated keywords with a particular media content item.By examining the keyword cluster, multiple different characters fromdifferent keywords may be combined together to generate trigrams.Keywords may include phrases describing a media content item. Any threeadjacently located characters in such a keyword phrase may be extractedto generate a character level keyword trigram. Different keywords and/orkeyword phrases may be combined to generate unique keyword phrases, fromwhich three adjacently located characters may be extracted. Similarly,word level keyword trigrams may be generated by combining such keywordsand/or keyword phrases to extract three different words. Certainstrategies may be used in combining keywords and/or keyword phrases togenerate character level and/or word level keyword trigrams. Forexample, keywords and/or keyword phrases that are contextually relatedto one another may be combined together for use in generating trigrams.The generated keyword trigrams may be associated with theircorresponding media content items. Such an association may be stored inthe media content item's metadata.

In step 614, it may be determined whether a voice command input has beenreceived from the user. Until such a user voice command input isreceived, steps 602-612 may be performed to associated differentkeywords and keyword trigrams with various media content items to enrichthese media content items' metadata for a voice command search. Keywordsand trigrams may be generated and associated with different mediacontent items continuously as new advertisements and new sources ofmedia content information become available. A search for media contentitems may begin when a voice command input is detected. The computingdevice may continuously monitor for any voice command inputs receivedfrom the user. Such voice command inputs may be received from amicrophone or another audio user interface device configured to deliveruser speech input to the computing device.

In step 616, in response to detecting a voice command input, a speechrecognized (SR) voice command text translation may be generated from thevoice command input. Once the voice command input is detected, the audioinput may be converted to a text transcript using speech recognitiontechniques. The text translation may be generated and saved in a memoryunit of the computing device for further processing.

In step 618, user utterance trigrams may be generated from the SR voicecommand input text translation. Once detected user voice commands aretranslated to a text transcript using SR techniques, the text transcriptmay be divided into smaller portions to separate different usercommands, resulting in user utterances. Such user utterances mayindicate a complete command or a portion of a user command. Userutterance trigrams may be generated from one user utterance or acombination of multiple user utterances. Word level and character leveluser utterance trigrams may be generated by examining such userutterances. Any three adjacently located characters in a user utterancemay be extracted to generate a character level keyword trigram.Different user utterances from the voice command text transcript may becombined to generate unique user utterance phrases, from which threeadjacently located characters may be extracted to generate a userutterance character level trigram. Similarly, word level keywordtrigrams may be generated by combining such user utterances and/or userutterance phrases to extract three different words. Certain strategiesmay be used in combining user utterances to generate character leveland/or word level keyword trigrams. For example, user utterances thatare contextually related to one another may be combined together for usein generating trigrams.

In step 620, keyword trigrams associated with media content items may besearched using the user utterance trigrams. Once it is determined thatthe user has issued a voice command to search for a particular mediaasset, keyword trigrams of different media content items may be searchedto determine the degree of match with the user utterances trigrams thatare generated from the user utterances related to media content searchvoice command. For example, a metadata database that includes keywordtrigrams associated with various different media content items may besearched for trigram matches. Multiple different user utterance trigramsmay be searched at the same time. The degree of match may increase basedon the number of keyword trigrams associated with a media content itemthat match user utterance trigrams. In such a media search, both wordlevel and character level trigrams may be searched. Character levelkeyword trigrams may be searched against character level user utterancetrigrams and word level keyword trigrams may be searched against wordlevel user utterance trigrams in any media search. Certain trigrams mayhave a higher weight attached to them, for example, based on theiruniqueness, and a match between such trigrams may increase a match scorethan a trigram match between trigrams having a lower weighting. Forexample, a trigram generator may assign weights to trigrams based on thefrequency of occurrence of such a trigram. If the trigram is not commonto other user utterances and/or media content keywords, then such atrigram may have a relatively high score assigned than more commonlygenerated trigrams. Accordingly, a match between trigrams having a highweight may result in a match having a high weighting. Multiple differenttrigram matches may be required between user utterance trigrams andkeyword trigrams. The match scores of each of the trigram matchesbetween user utterances for a voice command search and the trigramsassociated with a given media content item may be computed, summed, andnormalized to determine an effective match score between a media contentitem and the voice command search.

In step 622, incorrectly spelled homophones may be detected in userutterances and/or keywords from an SR text translation of the voicecommand input and/or advertisement audio, respectively. As voicecommands and audio signals of advertisements are translated to textusing SR techniques, certain words may be spelled with the spelling oftheir homophones. For example, the word “Rezurection” from the comedyshow “Ali G Rezurection” may be spelled using the spelling of itshomophone “Resurrection.” Once such a word has been spelled using thespelling of its homophone, the system may detect that one or morekeywords from the audio transcript of the advertisement or a userutterance may be incorrectly spelled. In some embodiments, such aspelling error may be detected during the voice command media search. Inother embodiments, such an error may be detected during text translationof an audio signal. Spelling errors in text translations may be detectedby comparing the keyword or user utterance to media information or othersources of information. For example, by parsing through webpages ormedia feeds for such a show, it may be determined that “Rezurection” isthe correct spelling and that the SR spelling of “Resurrection” isincorrect. For example, “Resurrection” may be determined to be incorrectif it is determined that the spelling “Resurrection” is not associatedwith that media content item in such webpages and/or media feeds. If nospelling error is detected, the method 600 may proceed to step 626 toidentify the media content item in the voice command search.

In step 624, in response to detecting an incorrect spelling in mediakeywords or user utterances, the incorrectly spelled homophones may beadjusted to prevent misleading search results. In some embodiments,after such a spelling error is detected, the associated keywords or userutterances may be corrected with the appropriate spelling. Trigramsresulting from such keywords or user utterances may also be correctedwith the appropriate spelling. Additionally or alternatively, theincorrectly spelled keyword or utterance and associated trigrams may bedeleted.

In some embodiments, no such detection for incorrectly spelledhomophones may be performed. However, in such embodiments, the incorrectspelling may not necessarily cause the search to render incorrectresults (e.g., returning the wrong media content item and/or notreturning any media content item as a search result). For example, as avirtue of performing a trigram search with multiple user utterancestrigrams being searched against multiple keyword trigrams associatedwith each media content item, a few spelling errors may notsignificantly affect the match score between the media content item andthe voice command input specifying the properties of the desired mediacontent item.

In step 626, a media content item matching the voice command may beidentified. Once any spelling errors, if detected in a text translation,are resolved, the media search may be completed and a resulting mediacontent item that best matches the user voice command input may beidentified. For example, the media content item with the highest matchscore between its associated keyword trigrams and the user utterancetrigrams (or an indication thereof) may be provided to the user as aresult of the trigram match process.

In step 628, the user utterance trigrams in the voice command input maybe associated with the media content item identified to match the voicecommand. Once a media content item has been identified to best match thevoice command input identifying the characteristics of the desired mediacontent item, the user utterances and/or the user utterance trigramscorresponding to such a voice command input may be associated with theresulting media content item. Such user utterances and/or user utterancetrigrams may be added to the metadata of the media content to enhancethe pre-existing database of keywords and/or keyword trigrams to providemore an improved media search in the future.

Aspects of the disclosure have been described in terms of illustrativeembodiments thereof. While illustrative systems and methods as describedherein embodying various aspects of the present disclosure are shown, itwill be understood by those skilled in the art, that the disclosure isnot limited to these embodiments. Modifications may be made by thoseskilled in the art, particularly in light of the foregoing teachings.For example, each of the features of the aforementioned illustrativeexamples may be utilized alone or in combination or subcombination withelements of the other examples. For example, any of the above describedsystems and methods or parts thereof may be combined with the othermethods and systems or parts thereof described above. For example, oneof ordinary skill in the art will appreciate that the steps illustratedin the illustrative figures may be performed in other than the recitedorder, and that one or more steps illustrated may be optional inaccordance with aspects of the disclosure. For example, in someembodiments, steps 602-612, steps 614-620, and/or steps 622-624 may beperformed simultaneously.

It will also be appreciated and understood that modifications may bemade without departing from the true spirit and scope of the presentdisclosure. The description is thus to be regarded as illustrativeinstead of restrictive on the present disclosure.

It is noted that various connections are set forth between elements inthe above description. These connections are described in general and,unless specified otherwise, may be direct or indirect; thisspecification is not intended to be limiting in this respect.

What is claimed is:
 1. A method comprising: determining, by a computingdevice, one or more keywords corresponding to an advertisement that isassociated with a media content item; associating the one or morekeywords with the media content item such that a plurality of keywords,associated with the media content item, comprises the one or morekeywords; receiving a search request indicative of a search keyword;determining, based on a search indicating an association between thesearch keyword and at least one of the plurality of keywords associatedwith the media content item, the media content item; and causing, basedon the search, output of an indication associated with the media contentitem.
 2. The method of claim 1, wherein the receiving the search requestfurther comprises receiving a speech input indicative of an audio signalcorresponding to the keyword of the plurality of keywords.
 3. The methodof claim 1, further comprising: determining one or more second keywordsbased on user voice input associated with the media content item; andassociating the one or more second keywords with the media content itemsuch that the plurality of keywords associated with the media contentitem comprises the one or more second keywords.
 4. The method of claim1, wherein the associating further comprises forming at least one n-gramfrom at least two keywords of the plurality of keywords.
 5. The methodof claim 1, wherein the search request is associated with at least oneof: an audio signal or a user speech input.
 6. The method of claim 1,wherein the determining the one or more keywords further comprisesdetermining, based on a segment indicator of a media content, that theadvertisement is a segment within the media content.
 7. The method ofclaim 1, further comprising: determining one or more second keywordsbased on one or more prior search requests associated with the mediacontent item; and associating the one or more second keywords with themedia content item such that the plurality of keywords associated withthe media content item comprises the one or more second keywords.
 8. Themethod of claim 1, wherein the media content item comprises a scheduledcontent item.
 9. The method of claim 1, wherein at least one of the oneor more keywords, or at least one of the plurality of keywords, is basedon: Internet content; online database content; social network content;online comments; user input; speech; or voice commands.
 10. The methodof claim 1, further comprising: determining one or more homophones ofthe one or more keywords corresponding to the advertisement; andassociating the one or more homophones with the media content item suchthat the plurality of keywords associated with the media content itemcomprises the one or more homophones.
 11. A method comprising:determining, by a computing device, one or more keywords correspondingto a first media content item that is associated with a second mediacontent item; associating the one or more keywords with the second mediacontent item such that a plurality of keywords, associated with thesecond media content item, comprises the one or more keywords; receivinga search request indicative of a search keyword; determining, based on asearch indicating an association between the search keyword and at leastone of the plurality of keywords associated with the second mediacontent item, the second media content item; and causing, based on thesearch, output of an indication associated with the second media contentitem.
 12. The method of claim 11, wherein the receiving the searchrequest further comprises receiving a speech input indicative of anaudio signal corresponding to the keyword of the plurality of keywords.13. The method of claim 11, further comprising: determining one or moresecond keywords based on user voice input associated with the secondmedia content item; and associating the one or more second keywords withthe second media content item such that the plurality of keywordsassociated with the second media content item comprises the one or moresecond keywords.
 14. The method of claim 11, wherein the associatingfurther comprises forming at least one n-gram from at least two keywordsof the plurality of keywords.
 15. The method of claim 11, wherein thesearch request is associated with at least one of: an audio signal or auser speech input.
 16. The method of claim 11, wherein the determiningthe one or more keywords further comprises determining, based on asegment indicator of a media content, that the first media content itemis a segment within the media content.
 17. The method of claim 11,further comprising: determining one or more second keywords based on oneor more prior search requests associated with the second media contentitem; and associating the one or more second keywords with the secondmedia content item such that the plurality of keywords associated withthe second media content item comprises the one or more second keywords.18. The method of claim 11, wherein the first media content itemcomprises an advertisement.
 19. The method of claim 11, wherein thesecond media content item comprises a scheduled content item.
 20. Themethod of claim 11, wherein at least one of the one or more keywords, orat least one of the plurality of keywords, is based on: Internetcontent; online database content; social network content; onlinecomments; user input; speech; or voice commands.
 21. The method of claim11, further comprising: determining one or more homophones of the one ormore keywords corresponding to the first media content item; andassociating the one or more homophones with the second media contentitem such that the plurality of keywords associated with the secondmedia content item comprises the one or more homophones.
 22. Anapparatus comprising: one or more processors; and memory storinginstructions that, when executed by the one or more processors, causethe apparatus to: determine one or more keywords corresponding to anadvertisement that is associated with a media content item; associatethe one or more keywords with the media content item such that aplurality of keywords, associated with the media content item, comprisesthe one or more keywords; receive a search request indicative of asearch keyword; determine, based on a search indicating an associationbetween the search keyword and at least one of the plurality of keywordsassociated with the media content item, the media content item; andcause, based on the search, output of an indication associated with themedia content item.
 23. The apparatus of claim 22, wherein theinstructions, when executed by the one or more processors, cause theapparatus to receive the search request via a speech input indicative ofan audio signal corresponding to the keyword of the plurality ofkeywords.
 24. The apparatus of claim 22, wherein the instructions, whenexecuted by the one or more processors, cause the apparatus to:determine one or more second keywords based on user voice inputassociated with the media content item; and associate the one or moresecond keywords with the media content item such that the plurality ofkeywords associated with the media content item comprises the one ormore second keywords.
 25. The apparatus of claim 22, wherein theinstructions, when executed by the one or more processors, cause theapparatus to associate the one or more keywords with the media contentitem by forming at least one n-gram from at least two keywords of theplurality of keywords.
 26. The apparatus of claim 22, wherein the searchrequest is associated with at least one of: an audio signal or a userspeech input.
 27. The apparatus of claim 22, wherein the instructions,when executed by the one or more processors, cause the apparatus todetermine the one or more keywords by determining, based on a segmentindicator of a media content, that the advertisement is a segment withinthe media content.
 28. The apparatus of claim 22, wherein theinstructions, when executed by the one or more processors, cause theapparatus to: determine one or more second keywords based on one or moreprior search requests associated with the media content item; andassociate the one or more second keywords with the media content itemsuch that the plurality of keywords associated with the media contentitem comprises the one or more second keywords.
 29. The apparatus ofclaim 22, wherein the media content item comprises a scheduled contentitem.
 30. The apparatus of claim 22, wherein at least one of the one ormore keywords, or at least one of the plurality of keywords, is basedon: Internet content; online database content; social network content;online comments; user input; speech; or voice commands.
 31. Theapparatus of claim 22, wherein the instructions, when executed by theone or more processors, cause the apparatus to: determine one or morehomophones of the one or more keywords corresponding to theadvertisement; and associate the one or more homophones with the mediacontent item such that the plurality of keywords associated with themedia content item comprises the one or more homophones.
 32. Anapparatus comprising: one or more processors; and memory storinginstructions that, when executed by the one or more processors, causethe apparatus to: determine one or more keywords corresponding to afirst media content item that is associated with a second media contentitem; associate the one or more keywords with the second media contentitem such that a plurality of keywords, associated with the second mediacontent item, comprises the one or more keywords; receive a searchrequest indicative of a search keyword; determine, based on a searchindicating an association between the search keyword and at least one ofthe plurality of keywords associated with the second media content item,the second media content item; and cause, based on the search, output ofan indication associated with the second media content item.
 33. Theapparatus of claim 32, wherein the instructions, when executed by theone or more processors, cause the apparatus to receive the searchrequest via a speech input indicative of an audio signal correspondingto the keyword of the plurality of keywords.
 34. The apparatus of claim32, wherein the instructions, when executed by the one or moreprocessors, cause the apparatus to: determine one or more secondkeywords based on user voice input associated with the second mediacontent item; and associate the one or more second keywords with thesecond media content item such that the plurality of keywords associatedwith the second media content item comprises the one or more secondkeywords.
 35. The apparatus of claim 32, wherein the instructions, whenexecuted by the one or more processors, cause the apparatus to theapparatus to associate the one or more keywords with the second mediacontent item by forming at least one n-gram from at least two keywordsof the plurality of keywords.
 36. The apparatus of claim 32, wherein thesearch request is associated with at least one of: an audio signal or auser speech input.
 37. The apparatus of claim 32, wherein theinstructions, when executed by the one or more processors, cause theapparatus to determine the one or more keywords by determining, based ona segment indicator of a media content, that the first media contentitem is a segment within the media content.
 38. The apparatus of claim32, wherein the instructions, when executed by the one or moreprocessors, cause the apparatus to: determine one or more secondkeywords based on one or more prior search requests associated with thesecond media content item; and associate the one or more second keywordswith the second media content item such that the plurality of keywordsassociated with the second media content item comprises the one or moresecond keywords.
 39. The apparatus of claim 32, wherein the first mediacontent item comprises an advertisement.
 40. The apparatus of claim 32,wherein the second media content item comprises a scheduled contentitem.
 41. The apparatus of claim 32, wherein at least one of the one ormore keywords, or at least one of the plurality of keywords, is basedon: Internet content; online database content; social network content;online comments; user input; speech; or voice commands.
 42. Theapparatus of claim 32, wherein the instructions, when executed by theone or more processors, cause the apparatus to: determine one or morehomophones of the one or more keywords corresponding to the first mediacontent item; and associate the one or more homophones with the secondmedia content item such that the plurality of keywords associated withthe second media content item comprises the one or more homophones.